Patcher: Patch Transformers with Mixture of Experts for Precise Medical Image Segmentation

We present a new encoder-decoder Vision Transformer architecture, Patcher, for medical image segmentation. Unlike standard Transformers, it employs Patcher blocks that segment an into large patches, each of which is further divided small patches. Transformers are applied to the patches within patch, constrains receptive field pixel. intentionally make overlap enhance intra-patch communication. The encoder cascade with increasing fields extract features from local global levels. This design allows benefit both coarse-to-fine feature extraction common in CNNs and superior spatial relationship modeling Transformers. also propose mixture-of-experts (MoE) based decoder, treats maps as experts selects suitable set expert predict label use MoE enables better specializations reduces interference between them during inference. Extensive experiments demonstrate outperforms state-of-the-art Transformer- CNN-based approaches significantly on stroke lesion segmentation polyp Code released facilitate related research. (Code: https://github.com/YanglanOu/patcher.git .).